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Abstract 


I examine a new generation of Career and Technical Education (CTE) models that has shifted 
from isolated courses to sequences of study that integrate academics and skills in specific 
career areas. I use data for a competitive grant administered by the California Department of 
Education (CDE) that incentives K-12 school districts to partner with community colleges and 
businesses to increase the career readiness levels of high school students. This study provides 
causal estimates of receiving large grants (1.e., up to 15 million dollars) to create aligned 
pathways by leveraging a natural experiment that occurs at the margin of grant receipt. Per 
pupil CTE expenditures increased by 21.7 percent at school districts that received the grant 
compared to unsuccessful applicants. Furthermore, dropout rates declined by 23 percent in 
districts receiving grants. The reduction in dropout rates appears to be concentrated for 
females and 11" grade students. The impacts for females may be related to design choices by 
grant applicants to focus on creating career pathways in traditionally female dominated 
sectors (e.g. health care support). The cost of preventing a single student from dropping out 
from this intervention is approximately 18,000 dollars compared to the present discounted 
value of a high school diploma of 300,000 dollars. 
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1. Introduction 

The economic and social consequences of failing to obtain a high school diploma are 
stark. The limited job prospects and low wages of drop outs led Cullen and coauthors (2013) to 
label dropping out of high school an “economic death sentence” (p.134). Outside of the labor 
market, dropouts are incarcerated at greater rates and experience a host of poorer health and 
social outcomes (Pettit & Western 2004; Cutler & Lleras-Muney 2006; Rumberger & Rotermund 
2012). High school graduation rates have stagnated since the 1970’s and gaps persist for blacks, 
Hispanics and males (Heckman & LaFontaine, 2010; Murnane, 2013). Despite the reality of only 
36 percent of young adults obtaining a bachelor’s degree, the majority of high schools in the 
United States focus on a college preparatory curriculum (Snyder & Dillow 2012; McFarland et 
al. 2017). This stylized fact coupled with others (e.g., increasingly dynamic workforce needs, 
the rising costs of attending college, and the challenges to college completion) have fueled a 
renewed interest in new models of Career and Technical Education (CTE) programs for high 
school students. 

This new generation of CTE models has shifted from isolated courses to sequences of 
study that integrate academic learning and skill-building in specific career areas (Holzer et al. 
2013). Three features distinguish these models from previous vocational programs: aligned 
career pathways; integration of academics and career preparation; and training in practical 
technical and employability skills (Holzer et al. 2013; Dougherty & Lombardi 2016). Proponents 
argue that coupling career preparation and training with a college preparatory curriculum is a 
dramatically underutilized approach that can meaningfully reduce social inequality by providing 


salient employment skills through multi-course sequences in high school that link to 
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postsecondary degree and training programs (Cullen et al. 2013; Holzer et al. 2013; Dougherty, 
2018). 

Recent studies on new CTE models find promising results in terms of improved high 
school completion, yet these studies focus on a small number of schools that require an 
application, specialize in CTE and are highly-resourced (see Kemple 2008, Doughtery 2018 and 
Hemelt at al. 2019). Given their small scale, select-student population and unique set of 
resources, these studies of CTE impact are useful proof of concepts but may not generalize to the 
broader population. Furthermore, prior descriptive work on CTE course taking has documented 
positive student selection into career pathways suggesting that Ordinary Least Squares (OLS) 
estimates will produce biased estimates (Dougherty 2016, Kreisman & Stange 2017). This study 
leverages a natural experiment that allows us to recover causal estimates in an at-scale policy 
environment using data from over 800 school districts in California. 

This study examines an innovative statewide competitive grant program, the California 
Career Partnership Trust (CCPT) that awarded funds to districts partnering with community 
colleges and local businesses to form articulated career pathways in high-growth sectors. The 
selection committee used a detailed rubric to rate applicants and a pre-specified threshold based 
on applicants’ plans and their capacity to implement career pathways. Using Regression 
Discontinuity (RD) designs, I show that at the margin, the applicants who score just above and 
just below the threshold are indistinguishable based on their baseline characteristics and dropout 
rates. This study design is uniquely suited to separate out whether underlying capacity, district 
characteristics or additional dollars for career pathway models drives impact. 

This study focuses on two important outcomes. First, I examine the effect of grant receipt 


on overall CTE spending. Successful applicants increased their per pupil district spending on 
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CTE by nearly 100 percent of the grant amount. This finding is consistent with extant literature 
in public policy of local governments using the proceeds of lump sum grants for their intended 
purpose (i.e., “flypaper effect’). Second, I examine the most salient outcome for an intervention 
aimed at increasing the career readiness and labor market outcomes for high school students, 
dropout rates!. The substantial 23 percent reduction in dropout was driven by reductions for 
female students and may have been mediated by district choices to provide pathways in 
traditionally female dominated sectors (e.g., health care). 

This study is organized as follows: in section 2, I discuss the theoretical considerations 
for career pathways and the prior literature in this area. In section 3, I describe the CCPT grant 
program, the plans of the applicants and characterize grant implementation. In section 4, I 
describe the data; and in section 5, I describe the RD estimation strategy. I present my results in 


section 6 and conclude in section 7. 


2. Theoretical Considerations and Prior Literature 

The empirical literature on the effect of career preparation and training suggests that 
participation in CTE are associated with consequential, though not transformative, improvements 
in student outcomes. Previous research has documented a correlation between participation in 
vocational coursework and increased wages and labor participation rates (Mane 1999; Bishop & 
Mane 2004; Meer 2007; Kreisman & Stange 2017). However, research on older models of CTE 


also note that students positively select into vocational courses (Meer 2007; Kreisman & Stange 


'T also examine student test score outcomes but privilege the dropout rates in this study because the reduction in 
dropout implies that the testing population is endogenous. A reduction in the dropout rate suggests that marginal 
students remain enrolled in high school, potentially lowering the average ability of tested students in treatment 
schools. However, I find no evidence suggesting there was a change in average test scores at the assignment 
threshold. Test score results are available upon request from the author. 
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2017). A defining argument for the career pathway approach is increased student engagement 
and completion because students are able to make a connection between academics and their 
future work prospects. Yet the evidence on increases in educational attainment related to 
vocational course-taking is mixed (Agodini & Deke 2004; Cellini, 2006; Neumark & Rothstein 
2006; Kemple 2008; Dougherty, 2018). This may stem from the inability of previous studies to 
account for student selection into vocational coursework and heterogeneous capacity across 
implementing schools and districts. The studies that are able to account for confounding factors 
find increased on-time high school completion (Dougherty 2018; Hemelt et al. 2019) and longer- 
run evidence of improved labor market outcomes (Kemple 2008). These three studies, however, 
represent highly-resourced small-scale implementations of model CTE programming and thus 


may not generalize when implemented at-scale under varying political and economic conditions. 


2.1. Theoretical Considerations 


The first key result of this paper is to demonstrate that the recipients of the CCPT grant 
increase their spending on CTE programming. The CCPT grant functioned similarly to a lump- 
sum grant, increasing the overall resources of recipient districts. Economic theory suggests that 
the provision of a lump-sum grant to a local government will result in a modest increase in 
spending on the targeted area. A lump-sum grant effectively increases overall income and 
residents may choose other spending priorities with their increased budgets. However, a large 
body of literature has documented larger increases in spending than would be explained by 


economic theory, known as the “flypaper effect.” This well-documented finding that money 


? The CCPT application required applicants to propose a budget that showed maintenance of effort (i.e., no decrease 
in CTE spending based on prior year spending). However, these requirements are often without teeth and an 
applicant could, in theory, decrease their CTE spending or slow intended increased in spending as a result of grant 
receipt. 
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tends to stick where it is directed includes instances where local spending increases by unity 
(Hines & Thaler 1995). 

The second key result of the paper examines the impact of districts spending additional 
CTE dollars on student outcomes. Exposure to career training during high school may affect 
student outcomes through several channels. First, this training can increase students’ technical 
and soft-skills which can improve their employment prospects and wages. Indeed previous 
research has documented a correlation between participation in vocational coursework and 
increased wages and labor participation rates (Mane, 1999; Bishop & Mane 2004; Meer 2007; 
Kreisman & Stange 2017). Second, linking classroom learning to the real world may increase 
student engagement as students are able to make a connection between academics and future 
work prospects. The evidence on increases in educational attainment related to vocational 
course-taking, however, is mixed (Agodini & Deke 2004; Neumark & Rothstein 2006; Cellini, 
2006; Doughtery, 2018). 

Persistence in high school is the most proximal outcome. CTE pathways may improve 
outcomes by addressing information asymmetries for students. If students lack information on 
the types of careers available to them as well as knowledge about the courses necessary to gain 
entry to those occupations, structured career pathway sequences will serve to better match 
students with opportunities in the labor market. Providing students with career pathways can also 
function as a behavioral nudge. Schools provide a default choice architecture by 
institutionalizing guidance counseling thus giving students an inferred recommendation for a set 
of high school courses (and career choices). Career pathways may also reinforce students’ social 


identity and increase their sense of belonging and engagement with schooling. 
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Career pathways may also simply change the mix of courses students enroll in; time spent 
in CTE courses results in fewer courses taken in other subject areas. This may result in higher 
graduation rates along with decreased student learning and/or effort. Nonetheless, these 
arguments suggest that for students on the margin of graduating, CTE programming has the 
potential to keep students enrolled in school longer and teach skills valued in the labor market. I 


now turn to the extant literature on interventions that examine the impact of CTE interventions. 


2.2. Relevant Literature 


The CCPT call for proposals required partners to develop CTE pathways that are similar 
to two models examined in the literature, career academies and Linked Learning. Both models 
combine academic content with career and technical-skill building within a career theme (e.g., 
health professions). These models also feature partnerships with local business to provide work- 
based learning opportunities, job shadowing and mentoring to students (Saunders and Chrisman 
2011). 

The most rigorous evidence comes from a randomized evaluation from the mid 1990’s of 
career academies (Kemple and Snipes, 2000; Kemple, 2008). The researchers found that 
although the intervention did not improve standardized test scores, participants were at lower risk 
of dropping out of school and more likely to graduate on-time (particularly at schools with 
greater levels of support). In the long run, Career Academy students’ experienced sustained 
earning gains (averaging 11 percent or $2088 more per year) compared to their counterparts in 
traditional high schools. The labor market gains were particularly concentrated among young 
men (a group that has experience earning declines in recent years). This study, however, focused 
on 9 oversubscribed high schools in primarily urban contexts. Career academies may be 


particularly beneficial when students have access to larger labor markets. Furthermore, these 
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schools, by virtue of being oversubscribed, signal to students and their family that they have 
unique capacity (1i.e., human capital) to provide high-quality services to students. 

The other evidence on the effectiveness of career pathways are mixed but use weaker 
study designs. In one study of a multi-district pilot program of Linked Learning pathways 
researchers use a propensity score matching design to account for student selection and find no 
difference in their measures of student engagement: attendance and suspension rates (Fitzgerald 
et al 2016). The authors find that although participants were more likely to complete a college- 
preparatory curriculum they were not more likely to enroll in postsecondary education. A study 
of an earlier-era career pathways program, Tech-Prep, funded by the federal government in the 
1990’s finds that participation diverted participants from 4-year to 2-year institutions with no 
impact on high school completion (Cellini 2006). 

Two recent studies examine the effect of contemporary career academies on student 
performance using credibly causal designs. Dougherty (2018) studies the impact of attending 
three oversubscribed application-based schools in Massachusetts that use a lottery to admit 
students while Hemelt and colleagues (2019) examine outcomes for attending one career 
academy high school in North Carolina. Both studies document selection into CTE programs: 
males, whites and students with lower baseline test scores are more likely to enroll in CTE 
programs while special education students and English learners are less likely to enroll. In terms 
of impacts, both studies find increased high school completion using the results of an admissions 
lottery to simulate a randomized control trial design. These findings have strong internal validity 
but represent a limited view of the effect of career pathways due to the small-scale and high 
fidelity nature of the intervention. In this study, I examine the potential of career pathway models 


to affect change at-scale. 
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The success of at-scale CTE reforms likely hinges on the fidelity of implementation. As 
noted by Kemple and Snipes (2000), there was significant heterogeneity in the level of support 
provided to students at the different Career Academies. Similarly, Linked Learning evaluators 
noted the wide range of implementation quality across schools (Guha et al. 2013; Fitzgerald et al 
2016). Thus, the ability to bring about meaningful improvements in student outcomes may have 
less to do with the particular traits of a career pathway program and more to do with institutional 
capacity to implement a high-quality program. In this study, I make two important contributions: 
I am able to isolate the effect of providing career pathways by comparing school districts with 
similar capacity to implement these reforms; and I use data from a large heterogeneous state 


implementing these reforms at-scale. 


3. California Career Pathways Trust 


3.1. CCPT Grant Overview 


The CCPT grant was established to help K-12 school districts, community colleges and 
employers build partnerships to help students transition into postsecondary education or the 
workforce in high-growth sectors. This competitive grant program, established by state law in 
2014, provided 500 million dollars in funding for career pathway programs. Staff in the Career 
and College Transition Division of California’s Department of Education (CDE) administered 
the application process across two round of funding in 2014 and 2015. Applicants were rated on 
a common rubric by multiple experts from CDE, the California Community Colleges 


Chancellors Office (CCCCO), and the California Workforce Investment Board (CWIB)?*. Grant 


3 Applicants were rated on seven pre-specified criteria and a rubric available in the application packet (available at 
https://www.cde.ca.gov/fg/fo/r17/documents/ccptl4app.doc). The scoring rubric consisted of seven criteria 
including a statement of need, description of the targeted population, the planned approach, partners, outcome 
measures, capacity and sustainability and budget. 
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recipients were required to document their use of awarded funds by reporting enrollment in 
career pathways and tracking grant spending in publicly available district financial summaries. 

The short-term nature of the grant (i.e., over three years) provided start-up funding and 
served as an incentive for districts to develop sequenced curriculum that was aligned to the 
postsecondary partner institutions. As such, grant funding was distributed over three years with 
50 percent of funds dispersed in the first year, 35 percent in the second and the remaining 15 
percent in year three. A maintenance of effort provision required recipients to supplement CCPT 
grant dollars with existing CTE funding streams (1.e., other local, state and federal sources) as 
detailed in the application’s budget narrative. 

The CCPT grant design and implementation focused on providing access to and supports 
for a range of districts. CDE provided guidance to potential applicants through workshops, 
webinars and by contracting a non-profit agency to provide technical assistance. CDE officials 
hosted four full-day application workshops in the four major regions of the state as well as two 
online webinars to provide an overview of the application and answer applicant questions. 
Additionally, the applicants could apply for three different tiers of grants (i.e., in the amount of 
600,000, 6 million or 15 million dollars) to account for the range in school district sizes across 
urban and rural areas. Individual school districts were also able to form multiple partnerships 
although agencies could only serve as the fiscal agent (i.e., lead agency) for one grant 
application. To facilitate partnerships of interested entities the CDE required interested 
applicants to submit a Letter of Intent (LOI) two months prior to the application due date. A list 
of entities submitting LOIs was posted publicly on the CDE website and all fiscal agents were 


required to submit an LOI to be considered for a CCPT grant. 
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The goal of the CCPT career pathways was to create multi-course sequences in target 
sectors in high school that aligned with courses of study at postsecondary partner institutions that 
culminate in credentials that are valued in the labor market. Applicants were required to use data 
from the Bureau of Labor Statistics (BLS) to identify high-growth industries and high-wage 
careers in their region to create vertically-aligned pathways with coursework at the high school 
and postsecondary levels. For each proposed career pathway applicants were required to specify 
a program of study detailing which courses and subject areas students would complete in 
secondary school (i.e., in grades 9 through 12) in addition to specific courses at partner 
institutions in years 13 and 14. The CCPT application rubric and webinar support materials 
describe successful applicants as those who build partnerships between employers, schools and 
community colleges to improve student transitions to higher education and employment 
opportunities. 

A hallmark of the CCPT application was its prescriptive nature that prompted applicants 
to use data to support their design choices. One unique feature of the application requirement 
was a form that detailed the planned course of study by grade level. This document required 
applicants to list both the CTE and academic coursework in secondary and postsecondary (i.e., 
grades 9 through 14) along with specific occupations requiring only a high school education, 
technical certifications or postsecondary degrees and their average wage levels.* For example, in 
the health care sector and career pathway of health care administrative services an applicant 
identified a four-course sequence including Medical Core, Medical Careers 1 and 2, and Nursing 


Assistant (CNA) training to be taken during high school. Occupations identified requiring only a 


4 While there was space in the application to list courses to be taken in years 15 and 16 of postsecondary education 
for advanced skill jobs these were not required for applicants to receive the highest score on the rubric. 
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high school diploma included medical record technicians and medical transcriptionists earning 
approximately 18 dollars per hour. The proposed career pathway continues at the local 
community college resulting in a Health Information Technology certificate that includes 
coursework in medical terminology, health care data analysis and medical quality management 
etc. This certificate qualifies applicants to work as health information managers/specialists 
resulting in hourly wages of 30 to 36 dollars. Applicants were directed to the BLS url on the 
application to obtain the necessary wage data. 

The CCPT application process guided interested districts in a planning process to identify 
high-growth sectors in their region and provided support in choosing career pathways to align 
with this growth. Applicants were expected to build on existing CTE offerings and match grant 
dollars with existing district funds. This competitive grant process incentivized local school 
districts to transform their existing CTE programs to models advocates believe can transform 


student outcomes. In the next section, I discuss the implementation of these new pathways. 


3.2. Implementation Evidence 


Implementation evidence from CCPT grantees comes from several sources: a legislative 
report commissioned by the legislature in the original bill securing the programs passage; a 2- 
year implementation study of grant winners by researchers affiliated with the agency charged 
with providing technical assistance to grant winners and my own analysis of CCPT grantees 
applications. Together these sources illuminate the lived experience of CCPT grantees and 
uncover how the design features of the application process resulted in a particular focus on 
health support occupations traditionally dominated by females. 

The legislative report provides useful information on how the grant winners utilized 


CCPT funding. The authors reviewed financial and enrollment data provided by grant winners; 
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surveyed site-leads and utilized state administrative data on student outcomes. The report finds 
that a majority of funds were used to develop sequenced curriculum (35 percent) and to align 
those pathways with postsecondary partner institutions (24 percent) (California Department of 
Education, 2017). An additional 16 percent of grant dollars were spent on hiring specialists to 
create Work Based Learning (WBL) opportunities that connect CTE pathway curriculum with 
local business partners. The most common pathway implemented by grant winners was in the 
sector of Health Sciences and Medicine with a total of 49 pathways. Other popular industries 
included Information Communication Technology (ICT); engineering manufacturing and 
design.° The report notes that student enrollment in Health Sciences and Medicine pathways was 
nearly twice as large as the second most popular pathway industry sector in information 
technology (1.e., 19,690 students versus 10,201). The CDE also surveyed grantee project leads on 
their career pathway configurations and found 90 percent offered a CTE course sequence, 53 
percent offered WBL opportunities and 40 percent of sites had programs of study affiliated with 
their postsecondary partners (CDE, 2017). This report suggests that grantees utilized the funding 
as intended and that the focus on the grant funding went to re-designing high school course 
curriculum. 

The lead organization in charge of providing technical assistance during the application 
process also conducted an implementation study of Cohort 1 applicants over two years. Jobs for 
the Future (JFF) is a national non-profit that advocates for programs and policies to increase 
college and career readiness and workforce outcomes of underserved populations. Researchers 


attended four convening’s for CCPT grantees, conducted a document review and interviewed 


>In Appendix Table 1, the proportion of applicants proposing each pathways are listed. Health Sciences was 
proposed by 53 percent of applicants whereas ICT were proposed by 41 percent of applicants. I find no 
discontinuities in the sectors of pathways proposed at the assignment threshold. 
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representatives from 20 grantee sites. The researchers also conducted multiday site visits at three 
grantee sites that received 15 million dollar grants. The report finds that grantees spent their 
funds on professional development to create and align career pathways, purchased equipment to 
maintain or expand CTE programs and to coordinate across partnership sites (McLaughlin et al. 
2017). The report notes that grant recipients focused on training staff to implement these new 
models of CTE through recognized professional training programs (e.g., Project Lead the Way, 
Linked Learning models). The findings at the selected sites suggests that the CCPT grant was 
used as “start-up” funding to train staff, develop plans and launch new pathways. 

The report authors also uncovered several noteworthy implementation challenges facing 
cohort | grantees. First, site-coordinators noted that lack of buy-in from district leadership and 
school boards had been a previous barrier to implementing and extending the reach of career 
pathways. The coordinators reported that grant dollars and grantee status (i.e., winning a 
competition) elevated the status of CTE programs for district policymakers. The significant 
infusion of dollars “legitimized” career pathways as a high school reform approach to address 
college and career readiness (p. 10, McLaughlin et al. 2017). Second, grantees struggled to find 
intermediary organizations to facilitate relationships with postsecondary and business partners. 
Many of the site coordinators in the first cohort were still looking for assistance in establishing 
work based learning opportunities, scheduling guest speakers and finding internships for students 
in the second year of the program (McLaughlin et al. 2017). Finally, coordinators reported 
difficulty finding qualified staff to implement data systems to track students served by pathways 
and their outcomes. In summary, many cohort | grantees report that CCPT funding allowed them 
to make progress in implementing the career pathway model by lowering political barriers but 


lack of qualified staff resulted in delayed rollout of crucial aspects of the program. 
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To supplement the available information on CCPT implementation I conducted a 
document review of the original CCPT applications. Two important findings result from this 
review. First, the short time line between the grant competition announcement and due date for 
letters of interest and first round applications may have resulted in more implementation 
challenges for the first cohort. Cohort | applicants had three weeks to submit their LOI’s from 
the competition announcement and another 6 weeks to devise a plan and submit it to the scoring 
agency. The announcement of the second round of competition was 13 months before the due 
date.° In addition to the compressed timeline faced by Cohort | applicants, the CCPT 
application underwent some revisions.’ The update application in 2015 required second round 
applicants to provide more specific information about the targeted pathway sectors including the 
roles and responsibilities of each partner schools, districts and businesses participating in a 
pathway. 

Furthermore, Cohort 2 applicants submitted three additional forms related to the design 
of pathways compared to Cohort | that required additional information on pathway plans. The 
differences in the timeline for planning and implementation in addition to changes to the 
application materials suggest that Cohort 2 had more planning time and improved technical 
assistance from CDE staff (i.e., application support and materials etc.).* Given these 
implementation details, I privilege estimates that examine the effects chronologically (1.e., by 


school year) rather than from time from grant application.” 


° There was also a 3 month lag between the submission of the LOIs and the application due date for the second 
cohort. This may have allowed additional districts to find partners who were interested in applying for CCPT. 

7 Personal communication with CDE staff on December 9, 2015. 

8 CDE contracted with an agency to provide technical assistance to both CCPT applicant cohorts and the agency 
may also have improved their ability to provide technical assistance over time. 

° In Appendix Table 2, I provide estimates of the treatment effect using a chronological and dynamic treatment 
effect approaches. The results support the implementation evidence that Cohort 2 experienced more immediate 
treatment effects while Cohort 1 effects appear after the first two full years of implementation. 


Page 15 of 29 


Sade Bonilla 
Stanford University 


In summary, the implementation evidence suggests that CCPT grantees used the funding 
as intended. The surveys and interviews conducted during the first two years of the grant suggest 
that most grant dollars were used to support pathway creation at high schools. Furthermore, the 
implementation evidence for Cohort 1, the first year of the grant was a planning year, consisting 
of professional development, curriculum development and the purchase of equipment to grow 
and sustain career pathways in the targeted areas. Another important takeaway comes from the 
design: applicants were instructed to focus on high-growth sectors and they accordingly focused 
on careers in the health care industry. Entry level positions in health care support occupations 
(e.g., medical assistants, certified nursing assistants etc.) are staffed by predominantly female 
workers. For example, 87 percent of workers in healthcare support occupations identified as 


woman in 2017.!° 


4. Data and Analytic Sample 


4.1. Data Sources 


The data come from several publicly available sources. First, staff at the CDE provided 
the CCPT applications for all applicants as well as the rating score used to determine funding 
status. The fiscal lead for each application can be a community college, school district, charter 
school, or county office of education. Most applications contained a list of the names of the 
secondary and post-secondary institutions and businesses involved in the partnership. I use these 
lists to locate publicly available data from the CDE and Common Core of Data (CCD) on K-12 
public schools and create a data set with district-level characteristics (e.g., race/ethnicity, 


free/reduced lunch, gender, enrollment etc.) from 2012-13 (1.e., the year prior to grant 


‘0 Retrieved from Table 11. Employed persons by detailed occupation, sex, race, and Hispanic or Latino ethnicity. Household 
Data Annual Averages, Department of Bureau of Labor Statistics https://www.bls.gov/cps/cpsaat1 1 .htm 
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announcement) to 2015-16. CDE also provides district-level financial data on resources and 
expenditures in an unaudited form using standardized account code structure (SACS). These data 
files identify spending on CTE broadly in addition to identifying specific CCPT related 
expenditures. Financial data is available from fiscal year 2013 through 2016. Other outcome data 
from CDE include subgroup level (1.e., by race/ethnicity and sex) information on dropout, 
truancy, chronic absence, suspension and expulsion counts. Furthermore, I supplement the 
available district characteristics with county level data from the Bureau of Labor Statistics on 


labor market conditions. 


4.2. Sample Construction 


There were 230 CCPT applicants across both rounds along with 765 district or county 
office of education partners, however, I exclude applicants for a variety of reasons. Charter 
schools for the most part do not report financial data separately to the SACS system nor are they 
represented in state data on dropouts and other outcome data. I therefore exclude the 27 charter 
fiscal lead applicants and their 10 district partners from the analytic sample.!! Next, I exclude 
three county offices of education that did not identify any district partners in their application 
and thus have no outcome data available.'* I further exclude 53 county offices of education who 
have no outcome data available but who have partner districts with available data. Finally, 85 
additional partners are omitted who lack baseline data on student demographics in the school 
year before applying for the CCPT grant. This results in a final analytic sample of 814 district 


entities representing 200 CPPT applications. 


'l Most of the excluded applications were single charter schools or groups of charter schools operated by the same 
charter management organization. 

County offices of education typically oversee specialized schools that serve special populations (i.e., incarcerated 
youth, hospital or homebound students etc.) and report data on the students they serve. The dropout rates reported 
are often over 100 percent and represent small populations in comparison to local education agencies. 
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We may be concerned that local education agencies and county offices of education who 
lack financial, demographic or outcome data is endogenous to their CCPT grantee status. It is 
important to note that missing data only threatens the internal validity of a regression 
discontinuity design if it discontinuous at the threshold for assigning treatment status. I run 
auxiliary regressions to test whether any type of missing data is discontinuous at the threshold 
and find no evidence to support this potential threat. 

In Table 1, I present descriptive statistics for this sample of 814 school districts. Just 
under one half of the sample applied during the first round of CCPT funding and over both 
rounds over 60 percent received funding. The majority of the grants awarded were at the 6 
million dollar and 15 million dollar level with just 5 percent of districts receiving the smaller 
600,000 dollar grants. The possible scores ranged from zero to 200 points though in practice they 
ranged from 28 to 184.5 points with a mean of 138 points. Prior to the announcement of the 
CCPT grant, districts spent approximately 226 dollars on CTE-related expenses per high school 
pupil enrolled in their district in 2013. There were broad increases in spending during the sample 
period due to increased funding from the state’s new Local Control Funding Formula (LCFF). In 
2015, the first and second year of CCPT funding for cohorts two and one, respectively, CTE 
spending per high school pupil increases by approximately 90 percent to 436 dollars per pupil. 

During the sample period the school level dropout rate at applicant districts declined from 
a mean of 2.51 percent of students in 2013 to 1.53 percent in 2017.'° District enrollment of high 
school students range in size from just 10 students to over 100,000 with the mean size district 


educating just under 8,000 students in traditional high school grades 9 through 12. The mean 


3 The school-level dropout rate is an event dropout rate consisting of all dropouts (i.e., numerator) in a particular 
school year divided by all enrolled students (i.e., denominator). 
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proportion of students eligible for the free or subsidized lunch is 51.7 percent of students but this 
ranges from nearly zero to almost all students. The average school district enrolls almost 47 
percent Hispanic students with whites averaging 34 percent of enrollment. Asians and blacks 
represent relatively smaller proportions of the public high school population in California at 8.8 
and 4.6 percent, respectively. The truancy rate, defined as a student having three or more 
unexcused absences of 30 or more minutes in one school year, consists of one-third of high 
school students. Just over 8 percent of students are suspended in a school year at the average 


school district. 


5. Analytic Strategy 


5.1. CCPT Application Scores 


Each partnership submitted an application that was scored using a common detailed 
rubric. Each application was read by three different professionals from CDE, the California 
Community Colleges Chancellor’s Office (CCCCO), or the California Workforce Investment 
Board (CWIB) and then averaged to arrive at the final score. In cases of large discrepancies 
between reviewer scores (i.e., more than 20 percent of total point difference) applications were 
assigned to a fourth reviewer, a manager in the Career and College Transition Division or a 
consultant. The fourth reviewers score was compared to the initial three scores and the outlying 
score was removed before averaging three scores !*. The cut score was based on the lowest 


amount of points applicants in the “outstanding” category on the rubric (e.g., 140 points in round 


'4 After personal communications with CDE I was unable to determine how many applications underwent the fourth 
reviewer rating process. CDE staff maintained it was “minimal.” My investigation of the continuity of baseline 
characteristics and the density of the rating score does not suggest manipulation. 


Page 19 of 29 


Sade Bonilla 
Stanford University 


1 and 150 points in round 2). The rating rubric contained seven sections that mirrored the 


required sections and forms of the CCPT application in each round. 


5.2. Identification Strategy 


The regression discontinuity approach relies on a relatively weak assumption that the two 
groups of applicants, those who scored one point above and belief the cut score, are equal in 
expectation except that one group received a large grant to create career pathways. In other 
words, in order to provide credibly causal estimates of increased funding for career pathways the 
only change at the margin of the CCPT grant award status is the probability of grant receipt. The 


estimation strategy can be described as follows: 


Yiq = BI(CCPT Score; = 0)+ f(CCPT Score;)+Xq+t + &ig (1) 


Here I(CCPT Score; = 0) is an indicator for whether the application score was above the 
minimum specified threshold for the applicant, i, and by extension their district partners, d. The 
variable, Xq represents district-level baseline characteristics, tT represents a vector of fixed effects 
for cohort and the grant level amount. Finally, €;g represents the error term, which are clustered 
at the applicant level. The estimand of interest, 6 represents the change in dropout status at the 
CCPT assignment threshold controlling for a function of the application score f(CCPT Score;). 
The preferred model allows for the slopes to vary on either side of the threshold and models a 
linear relationship between the application score and dropout outcomes. This preferred model is 
guided by a visual inspection of the data and Akaike’s Information Criteria (AIC) (Cook et al. 
2015). 

The literature on regression discontinuity designs is evolving as to the recommended 


estimation strategy, weighting structure and bandwidth selection. I check a variety of 
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specifications to provide a complete view of the estimated effects in addition to visual inspection 
of the data to inform our intuition around bandwidth selection and functional form. Following 
the literature, I estimate local linear regressions using observations in smaller bandwidths 
successively closer to the CCPT grant assignment threshold (Lee and Lemieux, 2010). I also 
examine estimates using the bandwidths calculated using the IK method from the procedure 
introduced by Imbens and Kalyanarman (2012). In this application the IK bandwidth procedure 
recommends using less than 20 percent of the full sample. Recent literature from Calonico and 
colleagues (2014, forthcoming) recommend bias-correction procedures that build on the data- 
driven bandwidth selections of the IK procedure and similarly use triangular kernel weights to 
emphasize the outcomes of observations close to the assignment threshold. In practice, these 
procedures produce similar results and I show a range of specifications and bandwidth 
restrictions to confirm the robustness of the RD results. 

The implementation evidence described above suggests that treatment may not have been 
uniform across the two cohorts. The literature on dynamic treatment effects suggests fitting 
treatment effects at different points in time (i.e., one year after treatment, two years after 
treatment etc.) rather than assuming constant effects. In this case, the timing of implementation 
for both cohorts varied. I estimate both dynamic treatment effects and effects based on calendar 
year for both cohorts, separately, and in stacked models with fixed effects for application year. I 


discuss these specification results in section 6 below. 


5.3. CCPT Treatment Status 


First, I show the first-stage relationship between CCPT grantee status and the assignment 
variable visually in Figure 1. The first-stage is virtually sharp, the change in probability of 


treatment status changes from nearly zero to | at the assignment threshold. The parametric 
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estimates in Table 2 confirm these findings ranging from 0.98 to 0.99 using quadratic 
polynomials and controls for application round year and application amount. One district, Los 
Angeles Unified School District (LAUSD), received a rating score below the threshold but 
received a CCPT grant. Personal communication with staff at CDE confirmed LAUSD’s grantee 
status and verified their original application score as being below the assignment threshold. I 
analyze all outcomes using an intent to treat (ITT) analysis based on the original assignment 
score rather than treatment status itself though these are highly related. LAUSD is the largest 
district in the state and I confirm the results by omitting LAUSD entirely and find qualitatively 
similar results. 

The first-stage relationship between the application score and the grant threshold clearly 
support a discontinuous jump at an arbitrary threshold dictated by CDE staff. Although the rubric 
was made available to applicants in the appendix of the application materials it is unlikely 
applicants could manipulate their overall rating score or known the exact cut off point to 
determine grant eligibility. The scoring process, however, was not blind and the three raters 
knew the identity of the fiscal agent and partnering district as these were part of the application 
evaluation. It is possible that raters may have had individual preferences. However, the process 
of having three raters independently rate applications without seeing the scores of the other raters 
limits the possibility that a single rater could have manipulated the assignment variable. 
Furthermore, the process of engaging a fourth reader for outlier scores reduced this possibility. 

We can further probe the necessary assumption that applicants are unable to manipulate 
their grantee status empirically. In Figure 2a, I present a histogram of the assignment variable 
using very small bin widths to examine potential heaping around the threshold. The histogram 


does not appear to contain heaps or observations concentrated just above or below the 
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assignment threshold. We can also look to the Cattaneo et al. density test (2017b, 2018) 
displayed in Figure 2b to test continuity around the threshold. This density estimator improves 
upon the McCrary (2008) density test by using a nonparametric approach that does not bin or 
transform the raw data. Figure 2b clearly demonstrates that there is no statistically significant 
jump in the density of the CCPT assignment variable at the threshold. 

Lastly, I examine the baseline characteristics of applicants in the 2012-13 school year for 
both rounds of applicants. !> In Table 3, I present evidence that baseline covariates are continuous 
at the assignment threshold using a two-stage procedure. In the first stage, I regress the outcome 
of interest (1.e., dropout rate for the relevant subgroup and year) on a linear function of all the 
baseline covariates. From this regression one can obtain predicted dropout rates that are an index 
of baseline characteristics (i.e., predicted dropout rates). This index is thus weighted by the 
influence of the baseline characteristics on the outcome of interest. We can then estimate an 
auxiliary regression of the index on the assignment variable to check whether the baseline 
characteristics are discontinuous at the threshold avoiding multiple comparison issues that arise 
with testing each baseline characteristic separately. !° 

In summary, both the institutional details and empirical evidence support the validity of 
the RD design for providing credibly causal estimates of CCPT grantee status. There is no 
evidence of manipulation of the forcing variable or discontinuities in baseline characteristics at 


the CCPT grant assignment threshold. Given these results supporting We now turn to the results. 


'S T choose 2012-13 because the CCPT call for applications was publicized as a two-round process during the 2013- 
14 school year and some districts may have intended to apply in the second round and changed their behavior prior 
to applying for the grant especially with regards to current levels of CTE spending. 

‘6 To be thorough, I also present the baseline characteristics in separate regressions in Appendix 3. 
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6. Results 

Increased grant funding should, in theory, increase the availability of career pathway 
spaces for students. Although districts are required to match CCPT funds with existing funding 
and report CCPT-related expenditures as a condition of grant receipt the literature suggests that 
these requirements are often poorly enforced. To investigate whether districts shifted their 
resources in response to grant receipt we can examine unaudited SACS data. In Table 4, I present 
parametric estimates the first-stage of per pupil CTE expenditures in columns (1) through (4). 
The estimates suggest that per pupil CTE expenditures in 2015 increased 45 to 65 dollars at the 
assignment threshold. The estimates are sensitive to functional form specifications and I use a 
visual inspection and AIC to determine the preferred model in column 3. As expected, per pupil 
CCPT grant funds increase at the assignment threshold and these results (in columns 5 through 
8) are statistically significant. No districts with CCPT assignment scores below the threshold 
reported spending CCPT dollars, except LAUSD. However, these results are qualitatively similar 
excluding LAUSD. In Figure 3, I present visual evidence showing an increase in per pupil CTE 
spending in 2015 at the assignment threshold. This result suggests a fly paper effect for CTE 
funding, that the districts used most of the intended funds for their stated purpose. 

Next, I show graphically in Figure 4 the impact of CCPT grantee status eligibility on 
dropout rates. Figure 4a shows no difference in 2016 years 1 and 2 of the grant. However, in 
2017 (1.e., years 2 and 3) there is a substantial decrease in dropout rates at the assignment 
threshold. The parametric estimates in Table 5 confirm the results in the figures. The reduction in 
dropout in 2017 is one-third of a percentage point an implied dropout reduction of 23 percent 
(i.e., 0.33/1.4). In Table 6, I show estimates with increasingly tighter bandwidths that confirm 


the graphical and parametric estimates. Although the bandwidths suggest by Calonico, Cattaneo 
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and Titunik (2014), and Imbens and Kalyanaraman (2012) are not statistically significant this is 
due to a reduction of power and increase in the standard errors of 150 percent. Importantly, the 
estimates remain negative implying reductions within the confidence interval of the full sample 
estimates. 

In Table 7, I provide additional estimates for subgroups of student by race/ethnicity and 
gender. The reductions in dropout rate are driven by reductions for females, whites and 11" and 
12" grade students. I estimate these models using Seemingly Unrelated Regressions (SUR) to 
allow the error structure to correlate across models to directly test whether grade level effects, 
gender or race/ethnic differences are statistically significant. The magnitude and statistical 
significance of the estimates using SUR are qualitatively similar to the effects shown in Table 7. 
Using the SUR framework I find that the estimated effects for 11" graders are statistically 
significantly different than those for 9" graders but find not differences for other grade levels. 
Finally, I am unable to reject the null hypothesis that the estimates for whites and Hispanics and 
males and females are the same. 

In the appendix I provide estimates and figures that report dynamic treatment effects. The 
figures and parametric estimates provide a broadly similar picture of treatment effects 
concentrated 2 years post-implementation and specifically for female students. Furthermore, by 
estimating effects separately for cohorts it becomes apparent that dropout reductions are evident 
for Cohort 2 in year two while Cohort 1 experiences reductions in year three. Stacking the 
cohorts provides increased statistical precision and confirms the reduction in dropout rates 
estimating effects by calendar year, when both Cohorts have had time to implement their planned 
career pathway programming. The dynamic treatment effects are similarly precise when stacked, 


however, year three outcome data will not be available until spring of 2019. 
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7. Discussion 

In this study, I explore a large-scale competitive grant program that supports school 
districts in creating articulated career pathways linking high school coursework to community 
college attendance and job prospects. I leverage unique application data to exploit a natural 
experiment that occurs at the margin of awarding grants to provide credibly causal estimates of 
the increased availability of career pathways on high school dropout rates. I provide convincing 
evidence that CCPT grant recipients increased their CTE spending after receiving the grant 
relative to non-grant winners. The RD estimates document substantial decreases in the dropout 
rate for all students that are driven by reductions for female students. These reductions may be 
driven by applicants’ choices to implement career pathways in fields that traditionally attract 
females (i.e., health care etc.). 

The available implementation information provide a unique look into how policy rollout 
affects student outcomes. The first cohort had limited time to create their planned reforms while 
the second cohort were able to plan over a longer period and benefit from the learned experience 
of the earlier adopters. Finally, previous studies have found male students to be more likely to 
self-select into CTE career academies (Dougherty 2018; Hemelt et al., 2019). The unique design 
of the CCPT required applicants to identify high-growth sectors based on available government 
labor market projects and wage-growth data. The health care sector entry level roles are 
traditionally dominated by female workers and this intervention appears to document increased 
school engagement amongst females at the implementing school districts. Understanding the 
importance of these design features will allow practitioners and policymakers to target certain 


populations. 
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Additional research is needed to understand the longer-term impacts of career pathway 
programs however the substantial reduction in dropout rates of over 20 percent is striking. This 
translates to a reduction in one student dropping out for 300 students served at a cost of nearly 
18,000 dollars. These calculations assume, however, that benefits of the grant dollars are only 
realized for the students served during this time period. In reality, the CCPT grant provided start- 
up funds to develop curriculum for pathways, train staff and purchase equipment. Thus, the cost 
per student served is an upper bound estimate. Notwithstanding, the present discount value of a 
high school diploma is estimated to be 300,000 dollars (Cullen et al. 2013). Career pathways 
may be a cost effective method of increasing students’ educational attainment and providing 


skills valued by the labor market. 
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Table 1. Descriptive Characteristics 


Variable Mean Std. Dev. Minimum Maximim 


Application Characteristics 


CCPT Grant Recipient (Year 1 or 2) 0.639 0.481 0 1.0 
CCPT Score (centered) 3.2 21.65 -83.3 43.3 
I(CCPT Score; 2 0) 0.636 0.481 0 1.0 
Round | applicant 0.473 0.500 0 1.0 
Round 2 applicant 0.527 0.500 0 1.0 
600,000 dollar grant 0.055 0.229 0 1.0 
6 million dollar grant 0.448 0.498 0 1.0 
15 million dollar grant 0.496 0.500 0 1.0 
Spending Characteristics 
Per pupil CTE spending, 2013 $226.39 241.73 0 1367.68 
Total CTE spending, 2013 $1,987,145 7,226,672 0 56,663,452 
Per pupil CTE spending, 2015 $436.36 426.18 0 2624.8 
Total CTE spending, 2015 $2,612,019 6,595,140 0 50,523,460 
Per pupil CCPT spending, 2015 $51.78 130.01 0 742.54 
Total CCPT spending, 2015 $235,389 627,091 0 3,621,278 
Qutcomes 
Dropout rate, 2016 1.53 1.57 0 14.78 
Dropout rate, 2017 1.39 1.49 0 15.07 
Baseline Characteristics 
Dropout rate, 2013 2.51 1.92 0 9.68 
County unemployment rate 9.87 3.24 5 24.5 
Enrollment grades 9-12 7,865 20,268 10 159,724 
Free/Reduced-Price Lunch 0.517 0.221 0.003 0.996 
Asian 0.088 0.109 0 0.643 
Black 0.046 0.061 0 0.363 
Hispanic 0.466 0.249 0 0.989 
White 0.344 0.231 0 0.881 
Truancy rate 33.90 20.78 0.4 85.9 
Suspension rate 8.26 5.26 0.0 28.6 


NOTES. Abbreviations: CCPT, California Career Pathways Trust. Sample includes 814 school districts with a CCPT 
assignment variable. Per pupil funding and dropout rates are top coded at 99th percentile. Behavioral, dropout and financial 
data retrieved from the California Department of Education (CDE) at https://www.cde.ca.gov/ds/dd/. Enrollment and 
student characteristics retrieved from the Common Core of Data (CCD). 


Table 2. First Stage RD Estimates, CCPT Grantee Status 
Dependent Variable: CCPT Grant Recipient (Round | or 2) 


Fuzzy RD 
Independent Variable (1) (2) (3) (4) 
I(CCPT Score; 2 0) 0.987*** 0.986*** 0.98 1*** 0.98 1*** 
(0.010) (0.011) (0.015) (0.015) 
N 814 814 814 814 
R? 0.989 0.990 0.990 0.990 
Linear spline yes yes yes yes 
Quadratic spline no no yes yes 
Controls no yes no yes 


NOTES. *** p < 0.001 ** p < 0.01 * p < 0.05 Applicant group clustered standard errors in 
parentheses. Each column contains a regression of the estimated effect of treatment status on 
the assignment variable. The controls contain fixed effects for round of application (i.e., in 
the 2014 or 2015 round) and grant level (i.e., 600,000, 6 million or 15 million dollars). 


Table 3. Auxiliary RD Estimates of Baseline Covariate Balance 


Dependent variable: 2016 Dropout index 2017 Dropout index 
Sample (1) (2) (3) (4) 
Overall 0.000 0.050 -0.042 -0.036 
(0.087) (0.113) (0.091) (0.122) 
Hispanic -0.070 -0.099 -0.104 -0.178 
(0.095) (0.118) (0.101) (0.138) 
White -0.017 0.025 -0.043 0.004 
(0.100) (0.133) (0.096) (0.125) 
Female -0.032 0.008 -0.034 -0.047 
(0.089) (0.116) (0.099) (0.134) 
Male -0.017 0.023 -0.067 -0.054 
(0.089) (0.118) (0.091) (0.122) 
N 814 814 814 814 
Linear spline Yes Yes Yes Yes 
Quadratic spline No Yes No Yes 


NOTES. *** p < .001 **p<.01*p<.05. Applicant group clustered standard errors in parentheses. Each cell 
contains the results of a two-stage regression: in the first stage the 2016 or 2017 outcome for the relevant 
subgroup is regressed on all 2013 baseline covariates (see Table 1 for a description of covariates) and a 
predicted drop out composite is generated. The predicted achievement composite is then regressed on I(CCPT 
Score; 2 0). 


Table 4. Reduced-form RD Estimates for CTE Spending 


Dependent Variable: 2015 CTE Per Pupil Expenditures 
Independent Variable (1) (2) (3) (4) 
I(CCPT Score; 2 0) 44.84 164.4* 65.61* 119.3** 


(50.68) (70.61) (29.81) (40.66) 
R? 0.113 0.154 0.493 0.501 


N 814 814 814 814 
Linear spline yes yes yes yes 
Quadratic spline no yes no yes 
Controls no no yes yes 


2015 CCPT Grant Per Pupil Expenditures 
(5) (6) (7) (8) 


59.40*** — 54.63***  40.76*** = 38.77%** 


(9.81) (14.00) (9.62) (13.96) 
0.155 0.157 0.257 0.260 


814 814 814 814 
yes yes yes yes 
no yes no yes 
no no yes yes 


NOTES. *** p < .001 **p<.01*p<.05. Applicant group clustered standard errors in parentheses. Each cell contains the result of a 
separate regression of the effect of (CCPT Score; > 0) on Career and Technical Education related per pupul expenditures. Akaike's 


information criterion (AIC) implied optimal order of polynomial is linear. 


Table 5. Reduced-form RD Estimates, Dropout Rate 


Dependent Variable: 


Independent Variable 
I(CCPT Score; 2 0) 
R 

N 


Baseline controls 


2016 Dropout Rate 


(1) (2) 
0.0648 0.0456 
(0.177) (0.166) 

0.008 0.6 

814 814 

no yes 


2017 Dropout Rate 


(3) (4) 
0:97 -0.333** 
(0.177) (0.153) 
0.013 0.498 

814 814 

no yes 


NOTES. *** p< .01 **p<.05 *p<.l. Applicant group clustered standard errors in 
parentheses. Each cell contains a regression of the dropout rate on I(CCPT Score; > 0), a 


linear spline of the assignment variable and baseline covariates. All models condition on a 


linear spline of the assignment variable. Akaike's information criterion (AIC) implied the 
optimal order of polynomial is linear. See Table 1 for a description of controls. 


Table 6. Reduced-form RD Estimates, Dropout Bandwidth Robustness Checks 


Dependent Variable: 2016 Dropout Rate 2017 Dropout Rate 
Sample () (2) (3) (4) 
est. n est. n 

Full Sample 0.0456 814 -0.333** 814 
(0.166) (0.153) 

|CCPT Score;| < 60 0.0494 809 -0.337%* 809 
(0.173) (0.161) 

|CCPT Score;| < 40 0.0455 768 -0.397%** 768 
(0.213) (0.197) 

|CCPT Score;| < 20 -0.0308 547 -0.657** 547 
(0.298) (0.258) 

|CCPT Score;| < 10 -0.755* 284 -0.670 284 
(0.436) (0.418) 

Kernel Weights -0.252 601 -0.685** 601 
(0.332) (0.293) 

CCT Estimates -0.399 207 -0.567 195 
(0.612) (0.593) 

IK Estimates -0.865 150 -1.289%** 150 
(0.478) (0.432) 


NOTES. *** p< .01 ** p< .05 *p<.1. Applicant group clustered standard errors in 
parentheses. Each cell contains a regression of dropout rates for districts within the specified 
bandwidth on I(CCPT Score; 2 0), a linear spline of the assignment variable and baseline 
covariates. Columns 2 and 4 contain the number of districts in the bandwidth. Kernel weight 
estimates estimate triangular kernel weights for observations within one standard deviation 
of the forcing variable. CCT are Calonico, Cattaneo and Titunik (2014) suggested 
bandwidths with kernel weights. IK are Imbens and Kalyanaraman (2012) suggested 
bandwidths with kernel weights. 


Table 7. Reduced-form RD Estimates by Subgroup 


Sample 2016 Dropout Rate 2017 Dropout Rate 
Subgroup (1) (2) (3) (4) 
Total 0.065 0.046 -0.270 -0.333** 
(0.177) (0.166) (0.177) (0.153) 
Hispanic 0.0522 0.0579 -0.201 -0.244* 
(0.200) (0.169) (0.174) (0.132) 
White -0.500 -0.498 -0.666** -0:5287** 
(0.382) (0.305) (0.265) (0.164) 
Female -0.007 -0.016 -0.280* -0.404*** 
(0.165) (0.158) (0.155) (0.146) 
Male 0.141 0.119 -0.253 -0.256 
(0.204) (0.198) (0.206) (0.170) 
Grade Level 
Grade 9 0.0486 0.191 -0.0630 -0.0187 
(0.143) (0.129) (0.156) (0.130) 
Grade 10 0.0161 0.0467 -0.213 -0.125 
(0.220) (0.206) (0.147) (0.148) 
Grade 11 0.0743 0.00881 -0.272 -0.368** 
(0.176) (0.188) (0.179) (0.182) 
Grade 12 0.276 0.260 -0.401 -0.561* 
(0.370) (0.305) (0.379) (0.300) 


NOTES. *** p< .01 ** p< .05 *p<.l. Applicant group clustered standard errors in 
parentheses. Each cell contains the result of a separate regression of I(CCPT Score; 2 0) 
on dropout rates for different subgroups. All models condition on a linear spline of the 
assignment variable and the full set of controls from the preferred specification in Table 
4. N= 814 school districts for all regressions. 


Appendix |. Reduced-form RD Estimates of Pathway Sector Offerings 


Dependent Variable: (1) (2) (3) 
mean 
Pathways Listed in Application 0.015 0.002 0.990 
(0.021) (0.015) 
Sector 
Health Sciences and Medicine -0.093 -0.307 0.538 
(0.117) (0.167) 
Technology 0.195 -0.014 0.412 
(0.116) (0.171) 
Logistics and Transportation 0.006 -0.007 0.161 
(0.093) (0.126) 
Energy and Environment 0.093 0.052 0.201 
(0.091) (0.139) 
Engineering, Design and Architechture 0.047 -0.401** 0.221 
(0.101) (0.140) 
Engineering and Manufacturing -0.008 -0.08 1 0.417 
(0.118) (0.170) 
Arts, Media and Entertainment -0.057 -0.226 0.186 
(0.100) (0.142) 
Business and Finance 0.042 -0.265 0.231 
(0.104) (0.164) 
Hospitatility and Tourism 0.131 0.019 0.156 
(0.088) (0.130) 
Public Services and Safety -0.010 -0.104 0.206 
(0.103) (0.152) 
Building and Construction 0.073 -0.082 0.176 
(0.090) (0.146) 
Education and Child Development 0.003 -0.027 0.106 
(0.067) (0.102) 
Agriculture 0.202 0.324* 0.261 
(0.107) (0.161) 
Linear spline yes yes 
Quadratic spline no yes 


NOLES. *** p < U.UUL ** p < U.0L * p < U.U5. Robust standard errors in parentheses. Each cell contains 
the result of a separate regression of I(CCPT Score; 2 0) on whether the applicant propsed sector pathways 
in their original application. N=201 applicants for whether pathways were listed in the original CCPT 
application. n= 199 for all other models indicating the plan to implement the particular pathways sector. 
All models contain fixed effects for round of application (i.e., in the 2014 or 2015 round) and grant level 
(i.e., 600,000, 6 million or 15 million dollars). The third column provides the average number of 
applicants proposing a particular pathways sector. 


Appendix 2. Reduced-form RD Estimates Dynamic Treatment Effects 


Cohort 1 Cohort 2 Stacked 
(1) (2) (3) (4) (5) (6) (7) (8) (9) 
Total Female Male Total Female Male Total Female Male 
Calendar Year 
2016 0.187 -0.174 -0.213 -0.177 -0.268 -0.0565 0.0456 -0.016 0.119 
(0.167) (0.167) (0.229) (0.301) (0.273) (0.366) -0.166 -0.158 -0.198 
2017 -0.249 -0.407** -0.0806 -0.400 -0.437 -0.349 -0.333** -0.404** -0.256 
(0.170) (0.203) (0.165) (0.286) (0.267) (0.327) -0.153 -0.146 -0.17 
Dynamic 
1 year after 0.101 -0.138 0.0745 -0.177 -0.268 -0.0565 -0.108 -0.106 -0.0899 
(0.149) (0.166) (0.178) (0.301) (0.273) (0.366) (0.167) (0.157) (0.188) 
2 years after 0.187 0.174 0.213 -0.400 -0.437 -0.349 -0.169 -0.232 -0.112 
(0.167) (0.167) (0.229) (0.286) (0.267) (0.327) (0.158) (0.153) (0.187) 
3 years after -0.249 -0.407** -0.0806 - - - - - - 


(0.170) (0.203) (0.165) 


NOTES. *** p< .01 ** p< .05 *p<.l. Applicant group clustered standard errors in parentheses. Each cell contains the result of a 
separate regression of I(CCPT Score; 2 0) on dropout rates for indicated subgroup and cohort. All models condition on a linear spline 
of the assignment variable and the full set of controls from the preferred specification in Table 4. n= 385 for cohort 1 and n=429 for 
cohort 2; N= 814 stacked panel. 


Appendix 3. Auxiliary RD Estimates of Baseline Covariate Balance, Individual Covariates 


Sample: 
Baseline Covariate 
Unemployment rate 


Per pupil CTE spending 
CTE spending (log) 
District enrollment (log) 
Free/reduced price lunch 
American Indian 
Pacific Islander 

Asian 

Hispanic 

Black 

White 

District dropout rate 
Truancy rate 


Suspension rate 


N 
Controls 
Linear spline 
Quadratic spline 


dQ) 


(2) 


Full Sample 


-0.037#*# 
(0.068) 
-0.040 
(22.858) 
-0.054 
(0.149) 
-0.004 
(0.017) 
0.050 
(0.018) 
-0.007 
(0.001) 
0.002 
(0.000) 
0.003 
(0.001) 
-0.003 
(0.004) 
0.009 
(0.001) 
0.000 
(0.004) 
0.053 
(0.127) 
0.095 
(2.564) 
0.021 
(0.462) 


814 
yes 
yes 
no 


-0.037#* 
(0.092) 
0.035 
(28.523) 
0.002 
(0.184) 
0.007 
(0.014) 
0.035 
(0.023) 
-0.019 
(0.001) 
0.010 
(0.000) 
-0.002 
(0.001) 
0.011 
(0.004) 
0.005 
(0.001) 
-0.010 
(0.004) 
0.019 
(0.165) 
0.144 
(3.657) 
-0.014 
(0.638) 


yes 


(3) 


1 SD (CCPT,) 


-0.043** 
(0.100) 
0.009 
(29.344) 
-0.019 
(0.183) 
0.005* 
(0.006) 
0.059 
(0.023) 
-0.028 
(0.001) 
-0.000 
(0.000) 
-0.001 
(0.001) 
0.003 
(0.005) 
0.010 
(0.001) 
-0.008 
(0.004) 
0.009 
(0.149) 
0.111 
(3.948) 
0.039 
(0.620) 


no 


(4) 


0.5 SD (CCPT,) 


-0.062*** 
(0.119) 
0.085 
(40.353) 
0.037 
(0.283) 
0.010**# 
(0.007) 
0.146 
(0.032) 
-0.017 
(0.001) 
0.006 
(0.000) 
0.006 
(0.002) 
0.015 
(0.004) 
0.012 
(0.001) 
-0.002 
(0.005) 
0.015 
(0.194) 
0.174 
(5.862) 
0.090 
(0.719) 


322 
yes 
yes 
no 


NOTES. *** p< .01 **p<.05 *p<.l. Applicant group clustered standard errors in 
parentheses. Each cell contains the result of a separate regression of the estimated effect of the 
baseline characteristic on the I(CCPT; = 0) and the indicated controls. The control variables 


included in these models are from 2012, the year prior to the baseline. Unemplomployment rate 


i measured at the county level, all other variables are measured at the district level. 
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Figure 1. First-stage, CCPT Grantee Status 


Notes. Graph of CCPT grantee treatment status by the assignment score (i.e., CCPT application score). Forcing 
variable centered at zero. Bin width 1.5 points, full sample. 
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Figure 2. Integrity of the Assignment Variable 


Notes. Assignment variable centered at zero. Full sample, bin width 0.25 points. Cattaneo, Jansson & Ma 
(2018) density test discontinuity estimate: 0.015 (.792). Standard error in parentheses. 
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Figure 3. First-stage, 2015 Per Pupil CTE Spending 


Notes. Graph of 2015 per pupil Career and Technical Education (CTE) expenditures by the CCPT 
assignment variable. Bandwidth +/- 40 points, Bin width 2.5 points. 
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(b) Dropout rate, 2017 
Figure 4. Reduced Form Results, Dropout Rates 


Notes. Graphs of the residualized dropout rates at in 2016 and 2017 by the CCPT assignment variable. Markers 
weighted by the number of districts in bin. Bin width 2.5 points. Bandwidth +/- 50. 
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(c) Female dropout rate, 2017 (d) Male dropout rate, 2017 


Figure 5. Reduced Form Results, Dropout Rates by Subgroup 
Notes. Graphs of the residualized dropout rates at in 2016 and 2017 by the CCPT assignment variable for indicated subgroups. Markers weighted 
by the number of districts in bin. Bin width 2.5 points. Bandwidth +/- 50. 
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(a) Dropout rate, Baseline 
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(b) Dropout rate, 1 year post 
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(c) Dropout rate, 2 years post 
Appendix 1. Dynamic Treatment Effects, Dropout Rates 


Notes. Graphs of the residualized dropout rates at baseline (e.g. 2013 or 2014), 

1-year post (e.g. 2015 or 2016), and 2-year post (i.e., 2016 or 2017) by the CCPT 
assignment variable. Markers weighted by the number of districts in bin. Bin width 3 points. 
Bandwidth +/- 50. 
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(c) Female dropout rate, 2 years post 
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(d) Male dropout rate, 2 years post 


Appendix 2. Dynamic Treatment Effects, Dropout Rates by Subgroup 


Notes. Graphs of the residualized dropout rates at baseline (e.g. 2013 or 2014), 1-year post (e.g. 2015 or 2016), and 2-year post (i.e., 2016 or 2017) by the CCPT 
assignment variable. Markers weighted by the number of districts in bin. Bin width 2.5 points. Bandwidth +/- 50. 
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